Object Detection, Classification, Tracking and individual Recognition for Sea Images and Videos

Manually monitoring the population displacement of fish species and the whale individuals is a painful and definitely unscalable process. Video data about fishes often require laborious visual analysis, moreover biologists often use photos of whale caudal for further analysis as it is the most discriminant pattern for distinguishing an individual whale from another. Therefore two challenges were announced in the SeaCLEF of LifeCLEF campaign, one for automatic fish categorization and enumeration, and another for automatic whale individual recognition based on visual contents. We elaborated a complex system to detect, classify and track objects (fishes) in underwater video by examining each image frame of it. We used Kalman filter to track the moving objects, and Hungarian method was used to match the pair of the objects in consecutive time periods because of many fishes. We categorized the detected fishes with C-SVC classifier, as an advanced SVM (Support Vector Machine) classifier. As further improvement we used color histograms and discriminant training method for filtering out false detections. For whale individual recognition we elaborated another system to compare the individuals by applying BoW model, during which Harris-Laplace detector and dense SIFT for creating low-level features. After that GMM based Fisher vectors were calculated and compared to each other with RBF kernel function. In addition to this we tried background segmentation as preprocessing.